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Abstract 

Background: Health services are subject to frequent changes, yet there has been insufficient research to address 
how staff working within these services perceive the climate for implementation. Staff perceptions, particularly of 
barriers to change, may affect successful implementation and the resultant quality of care. This study measures staff 
perceptions of barriers to change in acute mental healthcare. We identify whether occupational status and job 
satisfaction are related to these perceptions, as this might indicate a target for intervention that could aid successful 
implementation. As there were no available instruments capturing staff perceptions of barriers to change, we 
created a new measure (VOCALISE) to assess this construct. 

Methods: All nursing staff from acute in-patient settings in one large London mental health trust were eligible. 
Using a participatory method, a nurse researcher interviewed 32 staff to explore perceptions of barriers to change. 
This generated a measure through thematic analyses and staff feedback (N = 6). Psychometric testing was 
undertaken according to standard guidelines for measure development (N = 40, 42, 275). Random effects models 
were used to explore the associations between VOCALISE, occupational status, and job satisfaction (N = 125). 

Results: VOCALISE was easy to understand and complete, and showed acceptable reliability and validity. The factor 
analysis revealed three underlying constructs: 'confidence,' 'de-motivation' and 'powerlessness.' Staff with negative 
perceptions of barriers to change held more junior positions, and had poorer job satisfaction. Oualitatively, nursing 
assistants expressed a greater sense of organisational unfairness in response to change. 

Conclusions: VOCALISE can be used to explore staff perceptions of implementation climate and to assess how staff 
attitudes shape the successful outcomes of planned changes. Negative perceptions were linked with poor job 
satisfaction and to those occupying more junior roles, indicating a negative climate for implementation in those 
groups. Staff from these groups may therefore need special attention prior to implementing changes in mental 
health settings. 
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Background 

Health services are complex organisations that are charac- 
terised by substantial and ongoing organisational change 
as a result of research innovations and service develop- 
ments. While there is common acknowledgement of the 
frequency of these changes, there has been little research 
that addresses how staff working in these services antici- 
pate and respond to innovation [1,2], and none that fo- 
cuses specifically on acute in-patient wards in the mental 
health services. An exploration of these issues in the men- 
tal health services is important because distinct social pro- 
cesses and contextual features exist here [3]. In particular, 
in-patients are affected either personally or vicariously by 
the Mental Health Act (2003), the legislation that allows 
enforced detention in the U.K, which can lead to enforced 
medication and conflict. These factors may present bar- 
riers to change that are setting-specific. In addition, men- 
tal health nurses on acute wards are also engaged in the 
social milieu of the ward for the duration of their shift 
with little time allocated for strategic planning, and are 
often expected to play a key role in delivering changes, 
making their role different from other health professions. 

Previous research shows that organisational climate, or 
how staff perceive and respond to the characteristics of 
their work environment, affects work attitudes in mental 
health services [3-6]. A demoralising climate, as well as 
poorly orchestrated changes, may increase staff negativity 
and resistance to change as well as lowering morale [4]. It 
is also likely that nursing staff perceptions of implementa- 
tion climate [7] may also affect other work attitudes. As 
observed by Kajermo et al., [8], many descriptive studies 
identify barriers to implementing evidence-based practice, 
but the strategic link between barriers, implementation 
and outcomes has yet to be adequately explored [3]. Des- 
pite nurses representing the majority of the workforce, 
perceptions of implementation climate have been under 
studied. The emotional responses of staff as a result of ser- 
vice changes have also had little attention. Understanding 
how staff perceive change may be of particular use since 
perceptions can influence the successful outcomes of 
changes [9]. 

We explore the relationships between staff perceptions 
of barriers to change and occupational status and job 
satisfaction to highlight how staff can be supported be- 
fore and during processes of change, to improve the 
chances of successful implementation. 

Why has change been a challenge? 

Mental health nurses perceive many objective barriers to 
change in acute ward settings, including limited resources 
{e.g., beds, staff), the process of bed management, poor/in- 
determinate leadership, and violence [10,11]. Despite these 
perceptions, wide-scale changes to services have been 
pushed through in recent years, driven by political and 



economic factors. However, there is an increasing body of 
evidence in healthcare showing poor uptake of innova- 
tions and evidence-based findings into practice [12-14]. 

During periods of change, there are reciprocal interac- 
tions between the process and those involved [15]. The 
literature shows that low levels of involvement in change 
and changes that are not perceived to be beneficial can 
negatively impact the workforce. Organisational change 
in the UK National Health Service (NHS) is often im- 
posed via a top down approach, which may not take into 
consideration the views of the majority of nursing staff 
Indeed, change process issues such as poor involvement 
in planning, implementation and control of the project 
have been highlighted as potential barriers to success 
[16]. Staff in leadership roles may view changes more 
positively than those in direct care roles because they 
form part of the consultation process [17,18]. In our 
study, we will expand on this theme in a new context - 
acute mental health wards - hypothesising that nursing 
staff occupying managerial roles will view change more 
optimistically than more junior direct care staff. 

In a study examining organisational justice, Haar et al., 
[18] found that those employees of a local government 
organisation who were likely to directly benefit from 
work-family policies viewed them more positively than 
those who were not affected, who saw them as unfair. 
We will develop this theme to illuminate how organisa- 
tional justice is perceived in mental health settings. 

Other studies linked to acceptance of change have been 
carried out [19]. In particular, 'personal resilience,' charac- 
terised by perceived control over proposed changes, opti- 
mism, and self-esteem, was a predictor of acceptance to 
change. Poor job satisfaction was an outcome for staff 
who were less open to changes (i.e., who perceive more 
barriers) [19], a finding that has been replicated in the 
wider healthcare literature [1,2]. Poor job satisfaction is 
not only damaging for the individual, but has organisa- 
tional consequences as it has been linked to low staff re- 
tention [20] . We will explore the link between perceptions 
of barriers to change and job satisfaction in nursing staff 
in acute mental health settings. 

Involving all staff in changes would present significant 
logistical problems. However, staff opinion might be 
accessed directly before changes are implemented using 
a measure that captures staff sentiment on the front line. 
In this study, we generate such a measure using a par- 
ticipatory method of measure development [21], which 
is known to improve participation and engagement. We 
believe that this participatory approach produces items 
that are of importance to the group under study [22,23]. 

What are the issues In measuring change? 

The findings from studies looking at private sector orga- 
nisations or U.S. healthcare organisations may not be 
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generalisable to U.K. healthcare organisations, which are 
funded through taxation and which have a constitutional 
aim to consider the best interests of the client group 
[24]. There are currently no psychometrically robust 
measures designed to capture nursing staff perceptions 
of barriers to change in acute in-patient wards, nor has 
the idea of staff's direct participation in measure devel- 
opment been explored in this context. There are several 
measures developed in healthcare that focus on the up- 
take of evidence-based practice in clinical areas by men- 
tal health providers and nurses [25-32]. However, there 
are no measures which focus on general changes in a 
mental health setting. 

In this study, we first develop a psychometrically robust 
measure specific to the context of acute mental healthcare 
that includes a qualitative exploration of the emotional re- 
sponses of staff in relation to themes of organisational un- 
fairness. Then we use the new measure to explore how 
characteristics identified in the organisational literature 
are linked to perceptions of barriers to change. Specific- 
ally, we explore: whether staff in more senior positions 
have more positive perceptions of barriers to change than 
staff working in direct contact with service users; and 
whether staff with negative perceptions of barriers to 
change also have poor job satisfaction. This will clarify the 
impact of change on nursing staff. 

Methods 

Ethical considerations 

Ethical approval for this study was awarded by a local 
NHS Research Ethics Committee (07/H0809/49). Staff 
were assured that confidentiality and anonymity would 
be maintained unless any concerns about patient safety 
arose during the process. All staff gave informed consent 
to participate and were paid five pounds for their time. 

Measure development 

The process of measure development occurred across 
three stages which are outlined in Figure 1. 

Qualitative phase: instrument development 

The participatory framework for VOCALISE was adapted 
from the User-Focused Monitoring method developed by 
Rose [21] and Rose et al., [33,34]. This model is ideal be- 
cause it promotes involvement from nurses as 'stake- 
holders' of their clinical areas. The core principle is to 
translate participant views into items to ensure strong 
item content validity. This is accomplished by maintaining 
a direct contextual link between the researcher and par- 
ticipant through shared experiences. This reduces the 
usual power differentials between interviewer and inter- 
viewee, resulting in data that more accurately represent 
participant views [35]. In this research, the shared experi- 
ence was being a psychiatric nurse. 



( ^ 

Individual interviews 
(instrument development group, N=32) 



Qualitative analysis and 
item generation 



Expert validation 
(2 expert validation groups, N=6) 




Figure 1 Qualitative phase: study design for instrument 
development and item generation. 



Item generation 

The criterion sample (N = 32) was drawn from a large 
London mental health trust and comprised eight individ- 
uals from each level of acute in-patient nursing staff 
(nursing assistants, entry level qualified nurses, senior 
nurses and ward managers). 

Each staff member participated in a 30-minute inter- 
view based on themes drawn from the literature and 
from a reference group that included two nurses, a ser- 
vice user researcher, and a clinical psychologist. As the 
concept of 'perceptions of barriers to change' is abstract, 
the interviewee was asked to consider a scenario in 
practice where a significant change to clinical practice 
had occurred. Successive thematic analyses [36] were 
conducted using a qualitative software package (NVivo). 
This allowed new themes to be incorporated into subse- 
quent interviews and ensured that the content domain 
was explored broadly, enhancing content validity. 

Themes with the highest numbers of references and 
which were therefore the most important issues to staff 
were included as items that also made use of partici- 
pants' language. Consideration was also given to themes 
that were expressed with vehemence. Independent raters 
compared how similarly they interpreted the themes and 
codes of one interview using a function within NVivo to 
minimize subjectivity and increase how reliably the data 
were interpreted. 

Two rounds of feedback from acute ward nursing staff 
were then used to improve the item wording. This ex- 
pert validation process also formed another reliability 
check for the accuracy of the final questionnaire items 
and ensured high face validity. At the end of the devel- 
opment process, VOCALISE contained 23 items, which 
were answerable on a 6-point Likert scale ranging from 
strongly agree to strongly disagree. 

Psychometric testing 

Figure 2 outlines the psychometric testing process based 
on the Health Technology Assessment programme 
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Criterion validity: (group 4, 
N=125) 



Figure 2 Quantitative phase: study design for psychometric 
testing. 



recommendations [37] and Streiner & Norman [38]. 
Analyses were conducted using STATA 11 and SPSS 
Statistics 20. The results of the psychometric tests were 
used to improve the measure, which meant that modifi- 
cations occurred during the analyses and the final num- 
ber of items was reduced. 

Sample and preliminary tests 

All mental health acute ward nursing staff were invited 
to participate in this phase (ward managers, senior 
nurses, entry level qualified nurses and nursing assis- 
tants). Four sets of questionnaire data were collected 
from these staff, from all four boroughs of a large 
London trust. Each set of data was for a different 
purpose: 

1. group one (N = 40): feasibility study 

2. group two (N = 42): test-retest study 

3. group three (N = 275): factor analysis and the 
internal consistency of any subscales 



4. group four (N = 125): exploratory analyses of the 
associations between staff perceptions of barriers to 
change and occupational status and job satisfaction 
(criterion validity) 

Preliminary tests included an intraclass correlation, 
which was computed to establish that 'between ward' 
variance was greater than 'within group' variance [5]. 
Random effects regression models, clustering on ward, 
were then used to explore the effects of all staff charac- 
teristics on VOCALISE. Random effects models were 
appropriate because the data were multi-level. Random 
effects models meet the assumption that differences in 
the data relate to subsets of the population (in this case, 
that groups were located within wards). They control for 
unobserved heterogeneity so that significant effects can 
be said to be true effects rather than those observed due 
to differences in the sample. This method of analysis 
was also appropriate because despite differences in the 
scores of individuals, within each ward, the distribution 
was roughly symmetrical, suggesting general consensus. 

Measures 

Perceptions of job satisfaction were measured using the 
Index of Work Satisfaction (IWS) [39]. VOCALISE cap- 
tured staff perceptions of barriers to change. Negatively 
phrased items were reverse scored so that higher scores 
indicated more negative perceptions. 

Assessing the psychometric properties of VOCALISE 

Feasibility and acceptability were assessed on the 23- 
item measure. For feasibility, two additional questions 
assessing whether the measure was easy to complete and 
understand were asked of group one (N = 40), and for 
acceptability, a further two questions were added to the 
assessment of group four (N = 125). 

Reliability 

Group two (N = 42) completed the questionnaire at two 
time points. Test-retest reliability was assessed on total 
scores using Lin's concordance coefficient [40] and 
paired t-tests to investigate whether consistent bias 
existed. Item reliability was assessed using weighted 
kappa coefficients [41] on individual item scores at the 
two time points. Generally, scores of 0.21 to 0.4 are said 
to indicate 'fair agreement', scores of 0.41 to 0.60 indi- 
cate 'moderate' agreement, and scores above indicate 
'substantial' agreement [42]. As data were often clustered 
on particular responses, this meant that a high kappa 
was not achievable irrespective of the level of concord- 
ance between the two time points. Therefore, kappa max 
was computed, which is the proportion of the maximum 
kappa attained [43]. The assessments were six to ten 
days apart, which was appropriate because the measure 
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is of individual perceptions, which are subject to change 
over time. Items were removed on the basis of poor test- 
retest reliabiUty. 

The internal consistency of the scale using Cronbach's 
alpha [44] was then investigated with items that had per- 
formed poorly on the basis of test-retest removed. 

Validity 

Construct validity was examined through an exploratory 
factor analysis on responses from group three (N = 275), 
following preliminary checks (Bartlett's test of sphericity 
and the Kaiser-Meyer-Olkin measure of sampling ad- 
equacy), to determine the degree of correlation in the 
data. 

Using data from group three (N = 275), the internal 
consistency of the subscales were tested using Cronbach's 
alpha [44]. 

Exploratory analyses of the effects of staff perceptions 
were undertaken to assess criterion validity on the data 
from group four (N = 125), using random effects regres- 
sion models, clustering on ward, to predict VOCALISE 
total scores in relation to two hypotheses: 

1. Staff perceptions of barriers to change are more 
positive in staff with higher organisational status 
[17]. The two groups were managerial staff (in the 
U.K., this includes ward managers and senior 
nursing staff) and direct care staff (in the U.K., this 
includes entry level qualified nurses and nursing 
assistants). 

2. Staff with positive perceptions of barriers to change 
will also have high levels of job satisfaction [1]. The 
two groups were staff with positive perceptions of 
job satisfaction and staff with negative perceptions of 
job satisfaction. 

The IWS scores were dichotomised to create the two 
groups, using an a priori point (171). This was the aver- 
age from four studies of nurses' job satisfaction, which 
used IWS [45-48]. Positive perceptions were therefore 
defined as scores of 171 or below. 

Results 

Qualitative phase: instrument development and item 
generation 

There was a good spread across the demographic criteria 
of age, ethnicity, borough and gender of participants in 
this phase (see Table 1). 

During the interviews, staff focused on changes that 
were large scale and intended as long term. Examples in- 
clude: the introduction of protected therapeutic interaction 
time for key workers and their clients, the introduction of 
the smoking ban in hospitals, and the introduction of 
the 'Knowledge and Skills Framework', which is an NHS 



framework designed to monitor and support the manage- 
ment of career development. Within these examples, all 
staff also described the effects of incremental local changes 
and daily ward changes. 

Data reduction produced seven over-arching domains: 
'communication', 'generation of ideas', 'outcomes of 
changes', 'resistance', 'strategy', 'support and monitoring', 
and 'team dynamic'. Themes with fewer than 20 references 
were not included. Items were constructed around percep- 
tions of the team/ward and perceptions of the self in rela- 
tion to change. During the inter-rater reliability exercise, 
the raters achieved an 87% level of agreement in their in- 
terpretation of the main themes in the data. Those who 
participated in the expert validation process confirmed 
that the content of the measure was relevant and that the 
content domain had been widely explored. 

The qualitative data illustrated the emotional responses 
of staff toward changes. Quotations that focus on the 
theme of organisational unfairness are presented in Table 2. 

The psychometric properties of VOCALISE 
Sample and preliminary tests 

Demographic information from the four sets of ques- 
tionnaire data collected from mental health acute in- 
patient nursing staff are described in Table 3. 

A one-way analysis of variance assessing the effect of 
ward on the VOCALISE scores showed that the between 
group variance was greater than the within group vari- 
ance, F(7, 114) = 3.23, p = 0.004; N = 122). This suggested 
that random effects models would be suitable. 

In this sample, two demographic factors, age (Coef: -5.67, 
S.E: 2.57, p = 0.03, C.I: -10.70 to -0.64) and occupational 
seniority, significantly affected staff perceptions of barriers 
to change when assessed using the total score. These fac- 
tors were therefore controlled in all following regression 
analyses where relevant. 

Feasibility 

The feasibility study (group one, N = 40) showed that 
VOCALISE was easy to complete (94% agreed) and easy 
to understand (100% agreed). All staff found that with 
minimal explanation the measure could be completed by 
self-report. Changes were made to item wording, and 
those with poor or loaded phrasing were rephrased using 
more neutral language. This was determined by the re- 
search team. 

Acceptability 

Of the 125 participants (group four): 84 (73%) thought 
the length of the questionnaire was about right; 23 (20%) 
enjoyed filling out the questionnaire, while 78 (67.8%) 
had neutral feelings; and 107 (93%) did not find com- 
pleting the items upsetting. 
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Table 1 Demographic characteristics of interview 
participants 



Instrument development 

Group 

N = 32 (%) 



Staff grade 


Nursing assistants 


8(25) 




Entry level qualified nurses 


8 25) 




Senior nurses 


8(25) 




Ward mangers 


8(25) 


Ethnic group 


White British/Other 


1 8 (56) 




*BME 


14 (44) 


Gender 


Male 


14 (44) 




Female 


1 8 (56) 


Age 


Mean (sd) 


38(9.29) 




Range 


26 - 55 



•Note. BME = b[acl< minority ethnic group. 

The Flesch reading ease score for the measure was 62.4% 
(the recommended score for standard documents is bet- 
ween 60% and 70%). The Flesch-Kincaid Grade Level score 
was 7.6 (the recommended score is between 7.0 and 8.0). 
No items were altered as a result of the acceptability study. 

Table 2 Qualitative quotations on organisational unfairness 

THEMES QUOTATIONS 



Reliability 

Test-retest reliability was assessed using data from group 
two (N = 42). Four items were unreliable with a kappa 
max below 0.39 and were dropped from the scale, leav- 
ing eight items with fair reliability (0.39 to 0.49), seven 
items with moderate reliability (0.50 to 0.56), and three 
items with substantial reliability (0.61 to 0.71). 

Concordance between the total scores was good 
(Total score, rho = 0.76). However, a paired f-test showed 
that there was a significant difference between the two 
time points (t = -2.10; p = 0.04; mean difference = -2; 95% 
C.I: -3.93 to -0.07). Test-retest reliability was therefore 
assessed according to staff group revealing that staff 
in direct care roles (N = 26) were likely to change their 
scores (t = -2.91; p = 0.008; mean difference = -3.12; 95% 
C.I: -5.32 to -0.91). The scores of those in managerial roles 
were stable (t = 0.35; p = 0.73; mean difference = 0.64; 95% 
C.I: -3.36 to 4.64; N = 11). The test retest reliability of the 
T statements and general statements was also assessed 
showing that over time, the general statements (t = -1.88; 
p = 0.07; mean difference = -1.24; 95% C.I: -2.59 to 0.10; 
N = 37) were more likely to change than the T statements 
(t=-1.50; p = 0.27; mean difference = 0.14; 95% C.I: -1.82 
to 0.27; N = 40). 



STRATEGY: Not having a choice 
feels negative 



RESISTANCE: Patients don't 
want to 



RESISTANCE: Unfair task 
allocation 



RESISTANCE: Blame and 
resentment 



Entry level qualified nurse: 'We felt it was an added burden we felt that we had no choice.' 

Nursing assistant: Though I do think there was an element of sort of Machiavellian sort of style management 
going on but I don't think he was being malicious and that's why I probably why I didn't complain if I felt that I 
was being really victimised then I would have done but I think he just had a very unrealistic idea about his 
project and I think he was you know his management style was quite you know quite sort of what the word is 
exactly patriarchal quite sort of authoritarian.' 

Nursing assistant: 'But like I say some patients just don't want that interaction some patients are too poorly to 
have the interaction.' 

Entry level qualified nurse: 'It came to a point where I think when most of the patients didn't want to engage....Of 
course we couldn't enforce them....So you go introduce yourself and say we'll be meeting at such and such a 
point when the time comes and say well don't I don't want to talk about anything....Which makes it difficult....And 
very disheartening as well at times.' 

Nursing assistant: 'people have said I went to university for three years if you expect me to make beds and 

clean You know I mean when you, you know you hear people saying oh about team, team work and all that 

you wouldn't know team work if you were hit by Man United in a bus.' 

Senior nurse: 'And the turnover of patients is quite high so there's always - so you would catch up the way we 
do it now - you would catch up and see your people in the morning and say hello - how are you - is there 
anything on your mind that you badly need to talk about? So they'll have like 5 or 10 minutes and that's about it 
and everything else is directed to the unqualified to do which is sadly not the right thing to so because you 
should really spend a bit more time with your patient but that's how - how we are doing it now.' 

Senior nurse: 'Yeah that's it and then it's difficult because the motivated people are like well why are we not 
doing this today and then you get that kind of lack of consistency....And I think then people get quite angry 
about that and yeah.' 

Nursing assistant: 'And initially they, they were happy with this. And there wasn't much resistance then it was 
when they actually seen us doing what we were doing. I think because we weren't always working long days. 
Our initial shifts were four days a week it was I think it was half eight in the morning to half six. And, and 1 think 
people thought that that was not fair basically. So what it boils down people would be quite resentful and resist 
it in terms of, well - it's jealous.' 

Ward manager: 'I think that you know people automatically feel really dumped upon actually, and because, as 
well, there is another thing about people not really being aware of the bigger picture - I think so they just see it 
as well, I'm being asked to do my job plus run a group everyday ' 
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Table 3 Demographic characteristics of participants 







Group 1 
N = 40 (%) 


Group 2 
N = 42 (%) 


Group 3 
N = 275 (%) 


Group 4 

N = 125 (%) 


Staff grade 


Nursing assistants 


13 (32.5) 


14(33) 


76 (27) 


41 (33) 




Entry level qualified nurses 


14 (35) 


15 (36) 


114 (41) 


57 (46) 




Senior nurses 


9 (22.5) 


8(19) 


42 (16) 


15 (12) 




Ward mangers 


3 (7.5) 


5(12) 


16(6) 


9(7) 


Ethnic group 


White British/Other 


16 (40) 


15 (36) 


88 (32) 


33 (26) 




*BME 


24 (60) 


26 (62) 


1 77 (63) 


89 (72) 


Gender 


Female 


1 9 (47.5) 


20 (48) 


159 (57) 


78 (62) 




IVIale 


21 (52.5) 


22 (52) 


1 1 0 (39) 


46 (37) 


Age 


Mean (sd) 


44.44 (9.07) 


39.5 (8.8) 


38.11 (9.83) 


39.72 (10.01) 




Range 


28 - 64 


24 - 61 


19-67 


22 - 67 



*Note. BME = black minority ethnic group. 



Cronbach's alpha [44] was used to test the internal 
consistency of the 19-iteni scale. The overall alpha was 
0.71, indicating acceptable reliability [49]. However, one 
item was dropped from the scale because it was poorly cor- 
related with the total, indicating poor internal consistency, 
leaving a final alpha of 0.75, and 18 remaining items. These 
items are presented in Table 4. 

Validity 

The VOCALISE measure was then subjected to an ex- 
ploratory factor analysis to determine whether the pat- 
terns of relationships amongst the items could be simply 
explained by more than one underlying construct [50]. 
This assessment was carried out on the 18-item mea- 
sure, using data from group three (N = 275; fully com- 
pleted N = 240). The correlation matrix showed weak/ 
moderate correlations in all items. Bartlett's test of 
sphericity was significant (p = <0.001), which suggested 
that the variables were adequately correlated for a factor 
analysis. The Kaiser-Meyer-Olkin measure of sampling 
adequacy was good (0.8). 

As part of the exploratory process of factor analysis, 
solutions were analysed using both orthogonal rotation 
using Varimax and oblique rotation using Promax, and 
included five, four, three and two factors. An oblique so- 
lution was preferred because it showed the clearest dif- 
ferentiation between factor loadings. In this solution, 
three factors were retained because they fell before a 
steep drop in the curve of the scree test [51]. The first 3 
factors accounted for 43.50% of the variance and the 
eigen values were 4.23, 1.99 and 1.61 

The item groupings from the factor analysis had ad- 
equate internal consistency [49]. Subscale one, charac- 
terised by a sense of 'powerlessness', had an overall 
alpha of 0.73; N = 262. Subscale two, which is suggestive 
of 'confidence,' had an overall alpha of 0.66, N = 255, and 



subscale three, which echoes feelings of 'de-motivation', 
had an overall alpha of 0.59, N = 261. 

In order to aid interpretation, ranges were developed 
to show whether the VOCALISE total and subscale 
scores were positive or negative. These ranges are de- 
scribed in Table 5. Ambivalence in this case indicates 
that staff found it difficult to decide between slightly 
agree and slightly disagree. 

Exploratory analyses of the associations between 
VOCALISE and occupational seniority and job satisfac- 
tion were then conducted to assess criterion validity. In 
group four (N = 125), the mean VOCALISE score was: 
62.24 (sd = 11.47; range 38 to 93). 

Occupational status 

Occupational status was significantiy associated with staff 
perceptions of barriers to change after controlling for 
age (Coef: -6.62, S.E: 2.59, p = 0.01; C.I: -11.69 to -1.55, 
N = 108, 8 wards). The predicted mean VOCALISE score 
in the managerial group was 56.72, and 63.34 in the direct 
care providers group. 

Job satisfaction 

Those with negative perceptions of the barriers to 
change also had poor job satisfaction after controlling 
for age and occupational status (Coef: 10.43, S.E: 1.97, 
p = 0.001; C.I: 6.58 to 14.30, N = 101, 8 wards). The pre- 
dicted mean VOCALISE score in the high job satisfac- 
tion group was 56.76, and the predicted mean score in 
low job satisfaction group was 67.20. 

Discussion 

Measuring perceptions of barriers to change 

VOCALISE emerged out of a novel method of measure 
development that emphasizes the participation of nursing 
staff as a stakeholder group within mental health settings. 
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Table 4 VOCALISE measure 



WO. ITEM 



1. When it comes to change, information is not circulated effectively 
on my ward. 

2. I feel confident when delivering new changes. 

3. My whole team is regularly consulted about new ideas for ward 
practices. 

4. I'm too busy to keep up to date with information about the 
changes that are happening on my ward. 

5. We can easily fit new changes in with our usual ward practices. 

5. I feel disheartened when others do not want to get involved in 
changes. 

7. I think that managing risk is more important than delivering new 
changes. 

8. Changes just increase my workload and make my life harder. 

9. It is not clear how all changes that we are asked to make will 
really benefit my ward. 

10. My teammates think that there is no point trying to implement 
some changes because they won't work 

11. I find it de-motivating when new changes do not take patients' 
wishes into account. 

12. I think that some staff would rather let others take the lead in 
making changes. 

13. When some staff stop engaging with planned changes resistance 
spreads through my whole team. 

14. I do not really understand how to deliver some of the changes 
that are suggested by the management. 

15. Changes are audited to increase their consistent delivery on my 
ward. 

16. I always challenge team members who are avoiding delivering 
new changes. 

17. Inadequate staffing prevents changes being successful on my 
ward. 

18. Poor leadership prevents changes happening on my ward. 



As a brief self-report tool, VOCALISE can be used to cap- 
ture staff perceptions of large scale changes to practice, 
concisely. VOCALISE is advantageous over previous mea- 
sures (such as OSC, ORC, BARRIERS, EPBAS) because it 
combines aspects of the organisational, social/emotional 
and psychological context with staff perceptions of bar- 
riers to change, rather than treating each construct in 
isolation [3,25,26,52,53]. This type of measure might 
therefore usefully inform decisions for innovation made at 
the organisational level, by highlighting the negative im- 
pact of change on individuals. 



The participatory method yielded items that were de- 
scriptive and evaluative, suggesting that both are impor- 
tant in allowing staff to express cognitive and affective 
responses to change. Further, the benefits of the participa- 
tory model used to develop VOCALISE are visible in both 
the breadth of the construct investigated and the rich con- 
tent of the items. The scale addresses barriers arising from 
ward/environmental factors as well as from social factors. 
The content of some items {e.g., eight, nine and eleven - 
see Table 4) are of interest because they exemplify feelings 
of organisational unfairness relating to work pressures, 
which appear to have arisen as a result of poor consul- 
tation around new changes. Although the idea of work 
pressure as a barrier to organisational readiness for change 
is not new [53], VOCALISE makes the link between work 
pressure and the way that innovation is carried out expli- 
cit, thereby specifically addressing perceptions of imple- 
mentation climate. These more focused features may have 
been missed using a non-participatory method, and item 
11, which addresses nurses' perceptions about the effects 
of change on service users, is a new development. The 
links between how staff experience change and the feasi- 
bility of new changes in clinical practice may therefore be 
exposed successfully using this type of method. 

The psychometric properties of the final measure were 
promising. VOCALISE showed acceptable agreement, 
according to Cohen's kappa [41], and the total score 
showed good test-retest concordance. A total of 13 staff 
(35%) altered their score by more than 5 points in the 
direct care group but not in the managerial group, which 
indicates their sensitivity to instability in the inpatient 
environment in mental healthcare rather than rater bias. 
Indeed, during the qualitative phase of this study, staff 
suggested that their views might alter depending on the 
day, the staff on shift, and the client group. In practice, 
this suggests that VOCALISE data should be collected 
during a prescribed time frame, and the score should 
represent an aggregate of perceptions during that time. 

Researchers in the field of implementation science 
have argued that items capturing 'implementation cli- 
mate' should not be value-based and should comprise 
the shared perceptions of the workforce using descrip- 
tive, rather than evaluative items that focus on the orga- 
nisations' implementation practices and policies [7,54]. 
Using descriptive statements that focus on the team ra- 
ther than the individual might increase within group 



Table 5 Interpretative ranges for VOCALISE 



Scale 


Items (total) 


Positive range 


Negative range 


Ambivalent range 


Midpoint 


Total scores 


1-18 (18) 


18 - 54 


72-108 


55 - 71 


63 


Powerlessness 


4,5,7,8,9,14,17 (7) 


7 - 21 


28-42 


22 - 27 


24 


Confidence 


1,2,3,10,15,15 (6) 


6- 18 


24 - 36 


19-23 


21 


De-motivation 


6,11,12,13,18 (5) 


5 - 15 


20 - 30 


16 - 19 


17.5 
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consensus [7], because individuals' views of themselves 
are likely to vary more than individuals' views of the 
team. However, in an environment where both positive 
and negative approaches to implementation exist, the 
use of purely descriptive statements is a challenge [55]. 
The participatory method used to develop VOCALISE 
generated eight T and ten general statements containing 
the issues that staff referenced most frequently and 
therefore felt to be the most important when it came to 
describing the barriers to delivering significant changes 
to practice. 

How statements of perception function as a means of 
measurement in an environment that is subject to dynamic 
changes on a daily basis is worthy of further clarification. A 
mixture of T statements as well as general statements may 
be of use because, as this study shows, T statements are less 
likely to change over time. There were also less missing 
data in the T statements, perhaps because staff found these 
items easier to answer. Further, despite a mixture of both 
descriptive and evaluative statements, aggregated scores 
were successful in reflecting a ward view, because there 
was higher consensus in each group than between groups. 
The internal consistency of the entire scale was good, and 
the internal consistency of the subscales was adequate. 

What are the components of perceptions of barriers to 
change? 

The exploratory factor analysis indicated three latent psy- 
chological dimensions. Factor one links unsuccessful 
change to components of acute ward working that staff 
perceive as beyond their control. This highlights a sense of 
powerlessness that may be linked to poor work-related au- 
tonomy or to how staff prioritise change. Factor two ap- 
pears to relate to staff confidence in how the process of 
change is managed. Factor three is characterised by a 
sense of disillusionment and loss of motivation that im- 
pacts on the team dynamic, perhaps creating a barrier in 
group commitment to the process of change. 

Are poorer perceptions of barriers to change related to 
other work issues? 

Other studies have shown that staff with higher organ- 
isational status [18,56] view changes more favourably 
than those in more junior positions. Further, job satisfac- 
tion has been linked to staff who express more open 
views about new changes [1]. These results were repli- 
cated with VOCALISE, which demonstrates criterion 
validity. 

Issues of consultation and fairness in relation to 
changes may be linked to organisational justice theory, 
which suggests that managerial staff perceive change 
more favourably because of the way that organisational 
outcomes are determined [56]. Staff in leadership roles 
are more likely to be involved in the planning stages of 



new changes and therefore have an increased sense of 
control and responsibility over them. The qualitative 
findings showed that direct care staff frequently de- 
scribed poor involvement in new changes, which sup- 
ports previous findings in the literature [17,18]. A lack 
of interest in innovation existed amongst the client 
group, which affected staff motivation to deliver some 
changes. This increased feelings of blame and resent- 
ment among the team, and also emerged in response to 
issues of unfairness as linked to innovation. In the quali- 
tative data, this theme was most frequently referenced 
by nursing assistants. 

As the qualitative data identified, the direct care staff 
also made links between powerlessness and feasibility is- 
sues, feeling a sense of injustice because they were not 
able to deliver changes if neither their clients nor they 
perceived much benefit. Emotional responses such as 
blame and resentment also emerged in response to un- 
fair task allocation, or low commitment to change by 
colleagues. This may have ramifications for the quality 
of care delivered as those in direct care roles are gener- 
ally those who deliver changes and who spend the most 
time with patients. 

More research is therefore required to explore whether 
emphasising the involvement of nursing staff in such 
changes might prove an effective method of managing 
changes more successfully. 

Other studies of staff views of organisational changes 
have emphasised psychological concepts such as work re- 
lated empowerment [1]; control over proposed changes 
[19,57]; motivation [1]; and levels of optimism and self- 
esteem, which have been combined into 'resilience' 
[19,57]. Context-specific variables that capture informa- 
tion about how staff regard the change process, such as 
the provision of information, staff involvement and social 
support, have also been found to influence openness to 
change [16,56]. As a measurement tool, VOCALISE is 
novel because it incorporates many of these concepts 
into one measure (powerlessness over proposed changes, 
confidence/optimism and de-motivation). The individual 
items identify barriers/openness to change and percep- 
tions relating to the change process as well as context spe- 
cific issues. 

Limitations and future work 

VOCALISE was designed for use in a randomized con- 
trolled trial to implement nurse-led psychological therap- 
ies on acute in-patient wards, to assess how staff respond 
to general changes. Although this was a project to imple- 
ment evidence-based practice, VOCALISE was not de- 
signed to rate clinician responses to the importance and 
relevance of evidence-based practice, specifically. 

The data for this study were collected within one men- 
tal health trust. This was a very large service covering 
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different demographic and geographically diverse areas, 
but this still limits generalisability. 

As the amount of variance explained by the factors in 
this study was low and some factor loadings were rela- 
tively low, a confirmatory factor analysis on a larger sam- 
ple would be useful to confirm the measure structure. 

Implications 

The participatory method used to construct VOCALISE 
revealed items which staff felt to be important. When 
considering complex concepts such as staff responses to 
innovation, this method proved effective in illuminating 
both staff perceptions of barriers to change but also 
some of the psychological and emotional effects of 
change on staff This stakeholder-focused method of 
measure development has potential in other areas of im- 
plementation science where the relationship between 
change and those involved are integral to the successful 
uptake of change. It may, for example, usefully illumin- 
ate service user responses to innovation or to assess the 
barriers in sustaining innovation. 

Although VOCALISE was developed to capture staff 
responses to significant, general changes in mental 
health settings, it could also measure local and project- 
based change. The scale was not specifically designed to 
assess incremental change, but the emotional and psy- 
chological aspects of the scale also reflect how staff ex- 
perience the ongoing nature of change in the NHS, and 
therefore some effects of incremental change might be 
usefully picked up by VOCALISE. The item content of 
VOCALISE could be generalisable to other health set- 
tings, but more work should be undertaken to assess 
how VOCALISE performs in different areas. 

The findings suggest that those members of the nurs- 
ing workforce who are in direct care positions are am- 
bivalent about new changes. This may explain why 
changes are difficult to embed, and staff motivation, con- 
fidence and feelings of ownership over new changes 
should be addressed to improve uptake. 

Conclusions 

VOCALISE might be used in future research to identify 
whether nursing staff are motivated and confident enough 
to deliver new changes, as well as providing an opportun- 
ity for staff to engage with the process of change. As each 
of the factors may predict whether change is likely to be 
successful, interventions may then be tailored to improve 
the likelihood of embedding of changes into existing sys- 
tems and practices. If required, strategies may be intro- 
duced to help embed new changes. A measure of this type 
could also allow the impact of changes on staff over time 
to be studied, which is important as low morale and nega- 
tive perceptions of the workplace will be detrimental to 
the quality of care provided. 
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